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ABSTRACT 


Subjects  were  required  to  simultaneously  perform  a 
two-dimensional  tracking  task  and  respond  to  a  set  of 
lights  with  toggle  switches.   Five  levels  of  difficulty 
and  two  stimulus  presentation  rates  were  involved  in 
the  secondary  task.   The  purpose  of  the  experiment  was 
to  examine  time -sharing  performance  of  experienced 
military  pilots  and  to  investigate  differences  in  per- 
formance by  pilots  of  different  types  of  aircraft. 
Analysis  of  the  data  collected  from  20  subjects  showed 
that  correlation  between  elements  of  a  complex  task  is 
weak,  performance  levels  drop  as  task  load  increases, 
and  there  was  only  one  difference  found  between  any  of 
the  pilot  type  groups.   Dual-crew  fighter/attack  jet 
pilots  took  significantly  longer  to  respond  to  stimuli 
when  time- sharing. 
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I.   INTRODUCTION 

Human  factors  engineering  has  been  defined  as  "that 
branch  of  modern  technology  which  deals  with  ways  of 
designing  machines,  operations,  and  work  environments 
so  that  they  match  human  capacities  and  limitations" 
(Chapanis,  1959).   An  alternative  definition  considers 
human  factors  engineering  as  "that  area  of  psychology 
primarily  concerned  with  personnel  subsystem  interaction 
in  a  man-machine  or  man- environment  system"  (Waldeisen, 
197^) •   In  the  past  two  decades  systems  designers  and 
engineers  have  come  to  recognize  the  importance  of  the 
human  operator  as  part  of  complex  systems  such  as  Naval 
air  weapons  systems.   Consequently  there  has  developed 
an  appreciation  for  the  necessity  to  consider  human 
operator  skills,  abilities  and  limitations  in  the  design 
of  systems  (Wilkens,  1973). 

A  pilot's  task  today  requires  a  role  as  a  complex 
system  monitor  and  decision  maker  in  addition  to  per- 
forming complex  control  movements.   The  more  advanced 
air  weapons  systems  designs  currently  being  produced 
have  necessitated  the  requirement  for  more  sophisticated 
and  qualitatively  faster  and  better  performance  on  the 
part  of  the  human  operator.   These  requirements  have 
fostered  the  inclusion  of  disciplines  such  as  human 
factors  engineering  as  a  viable  input  to  complex  system 


development.   However,  a  number  of  problems  have  surfaced 
in  an  effort  to  apply  human  factors  engineering.   A 
central  issue  which  has  received  little  attention  is 
the  problem  of  methodology  necessary  to  measure  complex 
system  performance  from  the  standpoint  of  human  operator 
efficiency.   Part  of  the  problem  originates  with  past 
tendencies  to  analyze  human  performance  one  variable  at 
a  time.   There  is  a  serious  question  concerning  the 
appropriateness  of  such  methodology  to  adequately  assess 
complex,  time-sharing  performance  as  found  in  complex 
man-machine  systems. 

Time-sharing  is  defined  as  "the  requirement  (of  a 
human  operator)  to  divide  his  attention  between  two  or 
more  sources  of  information"  (Gabriel  and  Burrows,  1968) . 
It  is  difficult  to  imagine  a  situation  which  does  not 
critically  require  time- sharing  performance  from  the 
pilot  of  an  aircraft.   The  simple  task  of  flying  a 
straight  and  level  path  involves  (a)  monitoring  of  several 
flight  instruments  and  systems  gages,  (b)  coordinated 
movement  of  arms  and  legs,  and  (c)  monitoring  radio 
information.   Operational  requirements  may  impose  greater 
pilot  task  loading  due  to  maneuvers,  weather,  and  a 
myriad  of  tactical  factors. 

One  can  best  appreciate  the  complexity  of  a  military 

pilot's  job  by  taking  a  novice's  view  of  the  cockpit  in 

any  military  aircraft.   The  wall-to-wall  array  of  gages, 

lights,  dials,  switches,  and  other  controls  can  appear 

to  be  awesome. 
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Gabriel  and  Burrows  (1968)  demonstrated  that  it  is 
possible  to  improve  complex  performance  (specifically, 
time-sharing  abilities)  in  pilots  through  training. 

An  understanding  of  a  pilot's  ability  to  time-share 
has  potential  applications  in  two  major  areas,  systems 
design  and  training.   Both  applications  depend  upon 
appropriate  methods  for  measuring  human  performance  in 
time-sharing  tasks.   Questions  about  crew  size,  mission 
requirements,  scenarios,  proficiency,  and  training  could 
be  studied  more  objectively  if  such  methodology  were 
employed. 
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II.      IVIETHOD 

A.      DESIGN 

All  subjects  received  the  same  taped  instructions, 
the  same  number  of  sessions,  and  all  treatment  conditions. 
Thus,  each  subject  acted  as  his  own  control. 

Subjects  were  categorized  in  four  groups  of  five 
subjects  each  according  to  the  type  of  operational 
aircraft  each  had  flown.   Group  one  was  composed  of 
single-person-crew  jet  pilots;  group  two  was  dual-person- 
crew  jet  pilots;  group  three  was  helicopter  pilots,  and 
group  four  was  composed  of  propeller-driven  aircraft  pilots 

Subjects  received  learning  sessions  in  both  the 
tracking  task  and  the  secondary  task,  which  was  comprised 
of  five  levels  of  difficulty.   Each  subject  in  each 
group  performed  a  time-sharing  task  consisting  of  the 
tracking  task  and  each  of  the  five  levels  of  difficulty 
of  the  secondary  task.   Each  subject  performed  the 
time-sharing  tasks  under  two  stimulus  presentation  rates. 

The  dependent  variables  for  the  study  were  tracking 
scores  in  each  of  the  three  different  modes  of  time- 
sharing, total  reaction  time,  and  efficiency  scores. 

The  two-dimensional  tracking  task  required  each 
subject  to  maintain  a  small  dot  of  light  close  to  the 
center  of  a  cathode  ray  tube  screen  by  manipulating  a 
non-moving  force  stick  (Figure  1) .   Each  scoring  run 
was  100  seconds  long,  and  each  subject's  score  per  run 
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Figure  1.   Test  Subject  Operating  Station. 
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was  the  total  number  of  seconds  that  he  could  keep  the 
dot  of  light  inside  a  ^-millimeter  square  at  the  center 
of  the  screen. 

The  secondary  task  in  the  study  consisted  of  responding 
to  one  of  five  lights  located  on  the  left  side  of  the 
display  panel  by  turning  the  appropriate  light  off, 
using  one  of  the  five  toggle  switches  located  next  to  the 
lights  (Figure  2) .   Five  possible  light/switch  combinations 
(levels  of  difficulty)  were  possible.   Figure  3  is  an 
illustration  of  these  five  combinations.   To  the  left  of 
the  figure  is  an  illustration  of  the  switches  and  lights 
on  the  display  panel.   The  figure  may  be  read  as  follows. 
Starting  in  the  second  column  the  light  sequences  are 
numbered.   The  first  column  is  the  switch.   In  light 
sequence  1,  switch  number  1  activates  (or  deactivates) 
light  A.   In  light  sequence  number  three,  for  example, 
switch  number  2  activates  light  D.   In  addition  to  having 
five  sequences  the  secondary  task  had  two  stimulus 
presentation  rates.   Under  stimulus  presentation  rate 
ten  (SPR10)  a  light  would  come  on  every  ten  seconds  or 
a  total  of  ten  times  per  run.   Under  stimulus  presentation 
rate  twenty  (SPR20)  a  light  would  come  on  every  five 
seconds  or  a  total  of  twenty  times  per  run. 

Subjects  were  given  three  learning  sessions  of  seven 
runs  each  wherein  they  performed  only  the  two-dimensional 
tracking  task.   Prior  to  commencing  the  first  session  the 
subject  heard  a  taped  explanation  of  testing  procedures 
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Figure  2.   Display  Panel. 
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(Appendix  A) .   The  entire  first  session  was  devoted  to 
familiarizing  the  subject  with  equipment  and  procedures 
and  to  learning  the  tracking  task.   The  only  difference 
in  procedures  during  the  second  and  third  sessions  was 
that  the  subject  was  not  required  to  listen  to  the  taped 
instructions  again.   An  indicant  of  a  subject's  performance 
level  after  learning  the  basic  tracking  task  was  obtained 
by  averaging  the  final  five  runs  in  each  of  the  second 
and  third  practice  sessions.   It  was  at  that  point  that 
each  subject  was  considered  to  be  performing  at  his 
asymptotic  level,  i.e.,  the  subject's  increase  in  learning 
reached  a  negligible  rate.   This  score  was  called  the 
mean  tracking  score  (MTS) .   The  MTS  was  used  as  a  base 
score.   Deviations  from  the  base  score  were  used  as 
indicants  of  the  effects  of  mean  tracking  scores  under 
time-sharing  conditions. 

Before  commencing  the  fourth  session  each  subject 
listened  to  another  tape  which  explained  the  secondary 
task  (Appendix  B) .   The  first  run  in  this  session  in- 
volved only  the  two-dimensional  tracking  task.   After 
the  subject  had  scored  in  the  range  of  his  MTS,  time- 
sharing testing  began.   During  each  of  the  ensuing  ten 
runs  the  subject  performed  the  tracking  task  and  the 
secondary  task  simultaneously,  i.e.,  in  the  time-sharing 
mode.   The  SPR/light  sequence  combinations  were  given 
in  a  predetermined  and  randomized  order.   The  sequence 
in  which  stimuli  were  presented  in  each  run  were  also 
predetermined  and  randomized. 
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Before  each  of  the  time-sharing  runs  began,  the 
subject  had  up  to  50  seconds  to  learn  the  light  sequence 
(level  of  difficulty)  for  the  following  run.   During  those 
50  seconds  the  subject  was  at  liberty  to  turn  lights  on 
and  off  with  the  toggle  switches  and  thus  learn  the 
light/switch  combination  for  the  following  run.   When 
he  was  informed  that  his  50  seconds  had  expired,  he  was 
required  to  place  all  the  switches  in  the  same  position, 
i.e.,  either  up  or  down. 

Five  tracking  scores  were  averaged  in  SPR10  and 
SPR20 ,  respectively,  to  obtain  the  subject's  mean 
tracking  scores  under  SPR10  (MTS10)  and  under  SPR20 
(MTS20).   Total  reaction  time,  herein  defined  as  the 
total  time  (seconds)  that  any  light  was  on  during  a 
run,  was  recorded  with  the  number  of  incorrect  responses 
made  during  the  run. 

Efficiency  scores  were  the  average  of  the  sum  of 
the  total  reaction  time  and  the  number  of  incorrect 
responses,  multiplied  by  a  constant  which  was  derived 
by  dividing  the  total  number  of  mistakes  made  during 
time -sharing  runs  by  the  number  of  time -sharing  runs 
completed.   The  constant  was  approximately  0.8.   These 
averaged  scores  became  the  efficiency  scores  under 
SPR10  and  SPR20 . 

B.   SUBJECTS 

The  subjects  for  the  experiment  were  twenty  male 
students  at  the  Naval  Postgraduate  School,  Monterey, 
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California.   The  subjects  were  all  designated  Naval 
aviators  with  exactly  one  tour  of  operational  experience. 
Subjects  ranged  in  age  from  26  to  31  years  and  averaged 
29.6  years.   The  subjects  had  an  average  of  1^25  total 
flight  hours  and  had  not  flown  operationally  in  an 
average  of  1 . 6  years.   Exactly  $0%   of  the  subjects  had 
combat  experience  as  pilots  (four  in  group  one  and  two 
in  each  of  the  other  groups). 

All  subjects  were  volunteers  and,  with  two  exceptions, 
students  in  the  Naval  Postgraduate  School  Aeronautics 
Curricula.   They  were  not  compensated  for  their  time, 
but  were  motivated  by  competition  between  groups  and 
among  individuals.   Tracking  scores  and  averages  were 
posted  periodically  in  a  central  area. 

Subjects  received  a  short  explanation  of  the  methods 
and  goals  of  the  experiment.   Subjects  asked  few  questions 
after  testing  began. 

C.   STIMULUS  AND  APPARATUS 

The  Research  Education  Device  for  Basic  Aeronautics 
(RED  BARON)  provides  a  two-dimensional  tracking  task 
and  a  set  of  five  lights  and  five  switches  to  measure 
pilot  performance  in  the  tracking  task  and  in  time- 
sharing wherein  test  subjects  perform  both  tasks 
simultaneously. 

The  RED  BARON  has  six  major  components.   These  are 
listed  in  Appendix  B. 

Figure  1  shows  the  test  subject's  station  which 

includes  a  salvaged  FJ  aircraft  ejection  seat,  a 
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non-moving  force  stick,  and  a  display  panel.   Closing 
a  door  to  the  test  subject  station  activated  a  ventilator 
fan  and  a  light  which  were  included  in  the  design  for 
test  subject  comfort. 

Figure  2  shows  the  display  panel,  the  green  scorer 
light,  which  illuminated  whenever  the  test  subject  was 
"on  target,"  and  the  blue  timer  light,  which  illuminated 
five  seconds  before  and  all  during  scoring  runs,  are 
located  in  the  upper  left  and  upper  right  hand  corners, 
respectively.   The  cathode  ray  tube  oscilloscope  is 
located  in  the  upper  center  of  the  panel,  and  the  two 
knobs  for  adjusting  the  pipper  position  on  the 
oscilloscope  are  located  on  either  side  of  the  airspeed 
display  (a  non- functional  vestige  of  previous  studies) . 
Arranged  vertically  on  the  left  side  of  the  scope  are 
the  toggle  switches  and  lights  (components  of  the 
secondary  task) . 

Figure  k   is  a  view  of  the  entire  test  station  in- 
cluding, from  the  left,  a  tape  deck,  an  analog  computer, 
the  three  counters,  the  experimenter's  switches  and 
lights,  and  the  test  subject's  station. 

Figure  5  is  a  closer  view  of  the  outside  test  station 
with  all  of  the  aforementioned  equipment. 

The  tape  recorder  provides  a  repeatable  test  signal 
which  is  displayed  on  a  five-centimeter  square  portion 
of  the  oscilloscope.   The  subject-operated  force  stick 
generates  a  signal  which  acts  to  cancel  the  random  output 
signal  from  the  tape,  moving  the  displayed  pipper  toward 
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Figure  ^ .      Test  Station 


Figure  5-   Experimenter's  Station, 


20 


the  center  of  the  grid.   The  object  of  each  test  subject 
in  the  tracking  task  is  to  keep  the  pipper  inside  a 
"scoring  grid,"  which  is  a  four-square-centimeter  area 
at  the  center  of  the  oscilloscope.   Illumination  of  the 
green  scorer  light  provided  the  indication  to  each 
subject  that  he  was  "on  target,"  i.e.,  within  the 
scoring  grid. 

Inputs  to  the  analog  computer  are  from  the  stick  and 
the  tape  as  shown  in  Figures  6  and  7-   The  scoring  princi- 
ple used  in  the  tracking  runs  is  based  upon  the  cumulative 
time  during  a  100-second  run  that  a  test  subject  could 
keep  the  pipper  within  the  scoring  grid,  i.e.,  the  total 
time  in  a  run  that  the  subject  could  cancel  the  pre- 
recorded tape  signal  using  stick  inputs. 

Figure  7  shows  how  the  horizontal  and  vertical  tape 
and  stick  signals  are  summed  independently.   These  signals 
are  then  amplified  by  a  factor  of  ten,  as  shown  in 
Figure  8.   The  amplified  signals  are  then  passed  through 
a  sign  changing  amplifier,  and  both  the  original  signal 
and  the  signal  with  the  reversed  signs  are  fed  to  diodes 
which  allow  current  flow  in  only  one  direction  when  a 
selected  voltage  is  exceeded. 

The  increase  in  signal  magnitude  was  required  to 
activate  the  diodes ,  which  require  a  minimum  of  one-half 
volt  before  passing  current.   The  sign  changing  amplifiers 
are  necessary  so  that  both  plus  and  minus  signals  will 
trigger  the  compensator,  which  is  biased  for  signals  of 
one  polarity. 
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A  bias  of  -0.5  volts  is  patched  to  the  output  side 
of  the  diode  bus  so  that  when  any  summed,  amplified 
signal  exceeds  this  level,  the  diodes  will  permit  the 
current  flow  to  the  signal  comparator  input  INI  terminal. 
An  input  of  -O.36  volts  is  patched  to  the  1N2  terminal 
of  the  comparator  to  provide  a  comparator  bias  signal 
base.   The  1N2  input  may  be  varied  to  adjust  the  size 
of  the  scoring  grid  on  the  oscilloscope. 

The  comparator  relay  connects  the  output  of  a  ten- 
hertz  oscillator  to  the  electronics  counter  so  that  when 
the  summed  signals  (either  horizontal  or  vertical) 
exceed  a  specified  level,  the  oscillator  signal  to  the 
counter  ceases.   Since  the  counter  records  the  ten-hertz 
oscillations,  the  electronic  counter  records  the  time 
that  the  summed  signals  (both  horizontal  and  vertical) 
are  within  the  scoring  area  to  the  nearest  tenth  of  a 
second.   The  comparator  relay  also  activates  the  green 
scorer  light  on  the  display  panel,  which  advises  the 
test  subject  that  he  is  on  target. 

The  entire  comparator  is  activated  by  a  function 
switch  on  the  analog  computer. 

Figure  9  is  a  wiring  diagram  which  depicts  the 
entire  apparatus  for  the  secondary  task.   By  properly 
selecting  the  position  of  a  toggle  switch  on  his  panel, 
the  experimenter  is  able  to  complete  a  circuit  which 
includes  a  light  on  his  panel,  a  light  and  a  toggle 
switch  on  the  test  subject's  panel,  and  the  Navy 
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universal  counter.   Whenever  any  circuit  is  completed, 
the  counter  accumulates  digits  at  a  rate  of  60  per 
second.   The  circuit  can  be  interrupted  by  the  test 
subject  if  he  moves  the  proper  toggle  switch  on  his 
panel,  thus  deactivating  the  circuit.   The  sequence 
mixer  at  the  right  of  Figure  9  allowed  the  experimenter 
to  select  the  appropriate  light/switch  sequence. 

D.   PROCEDURE 

All  testing  was  completed  in  four  sessions  with  each 
test  subject.   During  the  first  three  sessions  (learning 
sessions)  each  subject  performed  only  the  tracking, 
task;  the  fourth  session  was  reserved  for  time-sharing 
performances . 

Before  the  first  session  began,  each  subject  received 
the  same  taped  instructions  (Appendix  A) .   He  was  given 
an  opportunity  to  ask  questions  before  the  session 
began  and  was  free  to  ask  questions  at  any  time  during 
the  session.   Each  of  the  first  three  sessions  was 
divided  into  seven  scoring  runs;  each  run  lasted  100 
seconds.   He  was  allowed  a  100-second  rest  period  be- 
tween each  tracking  run. 

Subjects  were  allowed  to  center  the  pipper  and  check 
the  size  of  the  scoring  grid  prior  to  each  run.   When 
the  subject  was  ready  to  begin,  he  so  informed  the 
experimenter,  and  then  waited  for  the  blue  timer  light 
to  illuminate.   Illumination  of  the  blue  light  was 
a  signal  to  the  test  subject  that  he  should  begin 
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tracking  because  the  comparator,  or  scoring  grid,  would 
be  activated  in  five  seconds.   The  subject  continued 
tracking  until  the  blue  light  went  out.   The  entire 
sequence  was  repeated  until  the  session  was  finished. 

The  second  and  third  sessions  (usually  completed  on 
separated  days)  were  the  same  as  the  first  session  except 
that  the  taped  instructions  were  not  repeated. 

During  the  first  three  sessions  the  experimenter 
recorded  the  tracking  scores  on  each  run.   The  scores 
of  all  subjects  were  open  to  public  scrutiny. 

Before  the  fourth  session  began,  each  subject  listened 
to  the  taped  instructions  for  the  secondary  task  (Appendix 
A).   He  then  completed  eleven  tracking  runs.   The  first 
run  was  in  the  tracking  task  alone,  but  each  of  the  last 
ten  runs  was  in  a  time-sharing  mode. 

Each  of  the  five  light  sequences  was  presented  twice, 
each  in  SPR10  and  once  in  SPR20 .   Figure  3  (explained 
in  the  Design  section)  is  an  illustration  of  the  five 
different  light/switch  sequences.   The  SPR/light  sequence 
combinations  were  presented  to  each  student  in  a  pre- 
determined randomized  order.   The  order  in  which  light 
stimuli  were  presented  in  each  run  was  also  randomized. 

The  first  run  in  the  fourth  session,  a  tracking-only 
run,  was  completed  to  ensure  that  the  subject  was 
capable  of  scoring  close  to  his  MTS . 

After  the  first  run  and  prior  to  all  time-sharing 
runs,  all  lights  on  the  display  panel  were  illuminated. 
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From  the  time  that  the  lights  came  on  each  subject  had 
50  seconds  to  familiarize  himself  with  the  light  and 
switch  sequence  for  that  run.   At  the  end  of  the  scoring 
run,  he  re centered  the  scoring  grid,  signaled  the 
experimenter,  and  waited  for  the  blue  light. 

The  scoring  grid  and  first  light  in  the  secondary 
task  were  illuminated  simultaneously.   The  second  light 
was  illuminated  either  five  or  ten  seconds  later  depending 
on  the  SPR. 

At  the  end  of  each  100-second  run  the  experimenter 
recorded  the  tracking  score,  the  total  reaction  time, 
and  the  number  of  incorrect  responses  made  during  the  run. 

A  new  light  sequence  was  then  selected,  and  the 
procedure  repeated  itself  until  the  session  was  completed. 
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III.  RESULTS 

Raw  data  were  reduced  as  follows: 
A-  =  Groups 

i=l  (Single-person-crew  jet  pilots) 

i=2  (Two-man  crew  jet  pilot) 

i=3  (Helicopter  pilot) 

i=Zf  (Propeller  aircraft  pilots) 
B-  =  Tracking  Score 

i=l  (In  SPR10) 

i=2  (In  SPR20) 

(Tracking  Score)i=(MTS)- (MTS)i+  100 

C-  =  Efficiency  Score 

1  s 

=  (1/5)   Z_( (Reaction  Time)  +  K  (#Incorrect  Responses)) 

i=l  (in  SPR10) 

i=2  (in  SPR20) 

K=  #incorrect  responses  made  in  all  runs 
Total  Time-sharing  Runs  Completed 

D-  =  (B.+C-)  =  Efficiency  Score  Plus  Tracking  Score 

(These  scores  were  normalized  to  a  mean  value  of 

100  and  a  standard  deviation  of  15  "before  they 

were  added.) 

Table  I  presents  the  analysis  of  variance  (ANOVA) 
summary  table  for  analysis  of  significant  differences 
in  performance  (on  the  tracking  task  scores  in  the 
time-sharing  modes)  between  pilot  groups  and  conditions 
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Source  Of 

Variance    Sum 

Degrees  of 
Freedom 

Mean 
Square 

F 

P 

A 

115.9 

3 

38.6 

1.294 

Insignificant 

B 

476.8 

1 

476.8 

15.968 

0.0001 

A  X  B 

24.6 

3 

8.2 

0.275 

Insignificant 

Block 

241.5 

7 

34.5 

Error 

746.5 

25 

29.9 

Total 

I605.3 

39 

TABLE  I:  4X2  Randomized  Block  Factorial  ANOVA  Summary 
Table  for  Analysis  of  Pilot  Type  Groups  (A)  and  Tracking 
Scores  In  Conditions  SPR10/SPR20  (B) . 
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SPR10/SPR20.   No  significant  differences  were  found 
except  due  to  SPR  conditions  (p  /o.OOl). 

Table  II  presents  the  ANOVA  summary  table  for 
analysis  of  pilot  type  groups  (A)  and  efficiency  scores 
under  conditions  SPR10/SPR20  (C) .   No  significant 
differences  were  found  except  due  to  SPR  conditions  (p  /  0.001) 

Table  III  presents  the  ANOVA  summary  table  for 
analysis  of  pilot  type  groups  (A)  and  combined  tracking 
and  efficiency  scores  under  conditions  SPR10/SPR20  (D) . 
No  significant  differences  were  found  except  due  to  SPR 
conditions  (p  /o.OOl). 

Table  IV  presents  the  ANOVA  summary  table  for  analysis 
of  pilot  type  groups  and  total  reaction  time  under  five 
levels  of  difficulty.   Effects  on  performance  due  to 
type  group,  conditions  of  difficulty,  and  the  interaction 
of  groups  and  levels  of  difficulty  were  significant. 
The  Duncan  multiple  range  test  for  difference  between 
groups  indicated  that  total  reaction  time  for  group  2 
was  significantly  longer,  than  for  the  other  groups 
(p^0.05).   (See  Figure  10).   The  Duncan  multiple  range 
test  for  total  tracking  scores  under  the  five  levels  of 
difficulty  indicate  that  each  level  was  significantly 
different  from  each  other  level  (p  <^0.05)  except  for 
level  2  and  level  5-   (See  Figure  11.) 

The  significant  interaction  between  groups  and 
levels  of  difficulty  are  represented  in  Figure  10. 
Group  2  was  the  slowest  at  all  levels,  with  the  difference 
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Source  of 

Degrees  of 

Mean 

Variance 

Sum 

Freedom 

Square 

F 

P 

A 

205.0 

3 

68.3 

2.737 

Insignificant 

C 

2757.3 

1 

2757.3 

110.480 

0.0001 

A  X  B 

60.3 

3 

20.1 

0.805 

Insignificant 

Block 

47-6 

7 

6.8 

Error 

623.9 

22 

25.0 

Total 

3694.0 

39 

TABLE  Hi   4X2  Randomized  Block  Factorial  AN0VA  Summary 
Table  for  Analysis  of  Pilot  Type  Groups  (A)  and  Efficiency 
Scores  Under  Conditions  SPR10/SPR20  (C) . 


Source  0 

f 

Degrees  c 

>f     Mean 

Variance 

Sum 

Freedom 

Square 

F 

P 

A 

1610.1 

3 

536.7 

1-778 

Insignificant 

D 

17871.8 

1 

17871.8 

59.22 

0.0001 

A  X  D 

184.7 

3 

61.6 

0.20 

Insignificant 

Block 

1937.9 

7 

276.8 

Error 

7544.7 

25 

301.8 

Total 

29149.2 

39 

TABLE  III:   4X2  Randomized  Block  Factorial  AN0VA  Summary 
Table  for  Analysis  of  Pilot  Type  Group  (A)  and  Efficiency 
and  Tracking  Scores  Under  Conditions  SPR10/SPR20  (D) . 
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Source   of 

Degrees   of 

Mean 

Variance 

Sum 

Freedom 

Square 

F 

P 

Total 

3532.9 

99 

Between 
Subjects 

129^.2 

19 

aorups 

527.8 

3 

175.9 

3.673 

0.05 

Error 
Between 

766.4 

16 

47.9 

AJithin 
Subjects 

2238.7 

80 

28.0 

Difficulty 
Levels 

803.7 

4 

200.8 

13.66 

0.001 

Diff.    Levels 

I  Groups            494.8 

12 

41.2 

2.8 

0.005 

Errors 

AJithin 

940.6 

64 

14.7 

TABLE  IV:  4X5  Mixed  Factorial,  Repeated  Measures  on  One 
Factor  for  Analysis  of  Pilot  Type  Group  and  Total  Reaction 
Time  Under  Five  Levels  of  Difficulty. 
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between  groups  being  the  greatest  at  the  most  difficult 
level  (level  4) .   Group  1  was  apparently  slower  than 
group  3  at  difficulty  levels  k   and  5«   Group  3  was 
apparently  slower  than  groups  1  and  4  at  level  2 ,  but 
faster  than  all  other  groups  in  level  4. 

The  present  results  may  be  suspect  due  to  a  rela- 
tively small  number  of  subjects  per  group.   Also  the 
group  differences  were  found  only  with  the  variable  of 
total  reaction  time  on  the  secondary  task.   No  group 
differences  were  found  with  efficiency  scores  or  tracking 
scores . 

Four  separate  rank  order  correlations  were  performed. 
The  rho  between  MTS  and  MTS10  rankings  was  0.386.   The 
rho  between  MTS  and  MTS20  was  0.364.   The  rho  between 
MTS10  and  C-,  was  0.486,  and  between  MTS20  and  Cp  was  0.124. 
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IV.   DISCUSSION 

The  analyses  of  variance  presented  in  Tables  I,  II, 
and  III  indicate  that  performance  under  SPR20  was 
significantly  degraded  as  compared  to  SPR10.   This 
finding  is  logical  in  that  one  would  expect  decreased 
performance  as  task  load  increased.   It  is  interesting 
to  note  that  there  were  no  significant  differences  due 
to  pilot  type  from  helicopter  to  propeller  to  fighter/ 
attack  pilot  groups. 

The  more  interesting  results  came  from  an  inter- 
pretation of  the  correlation  coefficients.   A  rho  of 
O.386  and  0.36^  between  MTS  and  MTS10/MTS20  respectively 
tend  to  indicate  that  13i%   of  the  variation  in  performance 
in  MTS  is  accounted  for  by  the  same  task  under  SPR10/ 
SPR20  when  performed  in  a  time-sharing  mode.   This 
indicates  that  the  ability  to  predict  performance  of  the 
same  group  on  tracking  time -sharing  performance  from 
tracking  non- time-sharing  performance  is  very  poor. 
The  findings  tend  to  support  the  contention  that 
methodologically  the  prediction  of  complex  performance 
elements  (such  as  tracking  scores)  should  not  be  attempted 
except  within  the  context  of  the  total  time-sharing 
task  environment.   Implications  may  extend  to  the  area 
of  complex  task  training.   Possibly  training  on  tasks 
such  as  tracking  should  be  conducted  within  the 
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total  time -sharing  task  context  (see  Gabriel  and 
Burrows,  1968) . 

The  rho  correlation  coefficients  between  MTS  10  and 
efficiency  scores  on  the  secondary  task  was  0.486.   The 
rho  between  MTS20  and  efficiency  scores  in  the  secondary 
task  was  0.124.   The  difference  in  correlations  tends 
to  show  that  the  more  complex  situation  of  MTS20  is 
less  predictive  of  efficiency  scores.   This  may  indicate 
that  the  more  complex  a  task  becomes,  the  greater  the 
necessity  to  measure  all  task  elements  in  a  time-sharing 
analog  to  the  real  world  task. 

It  would  be  of  interest  to  conduct  studies  with  a 
secondary  task  which  is  more  operationally  relevant  to 
time-sharing  with  tracking  tasks.   Further  areas  of 
interest  would  be  studies  of  portions  of  real  world 
flight  tasks  which  require  time-sharing  of  more  than 
two  tasks.   The  present  experiment  could  be  continued 
with  changes  in  the  type  of  stimulus  used  in  the  secondary 
task.   The  effects  of  environment  (heat,  noise,  vibration, 
etc.)  on  time-sharing  performance  could  be  studied.   Ex- 
perimenters could  also  evaluate  the  transfer  of  complex 
performance  training  in  a  simulator  to  actual  system 
operation.   The  more  ambitious  experimenter  could  develop 
methodology  for  dual  or  multiple  person  crew  complex 
performance  testing. 
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APPENDIX  A 
•  TAPED  INSTRUCTIONS  FOR  TEST  SUBJECTS 

Preliminary  Phase. 

"You  are  seated  in  the  Research  Education  Device 
for  Basic  Aeronautics,  also  known  as  the  RED  BARON. 
The  only  equipment  that  you  need  to  be  familiar  with  at 
this  time  are  the  non-moving  force  stick  on  your  right, 
the  oscilloscope  screen  in  front  of  you,  the  blue  and 
green  lights  on  the  upper  portion  of  the  display  panel, 
and  the  two  adjustment  knobs  on  either  side  of  the 
airspeed  display.   Do  not  place  your  fingers  behind  the 
display  panel;  there  are  some  exposed  wires  there  which 
could  shock  you. 

There  is  a  green  dot  of  light  on  the  oscilloscope. 
When  the  scoring  run  begins,  it  will  move  around  the 
scope  in  a  random  track.   Your  mission,  should  you 
decide  to  accept  it,  will  be  to  keep  the  dot  on  a  target 
that  is  located  in  the  center  of  the  oscilloscope.   You 
will  be  on  target  whenever  the  green  light  is  lit.   You 
will  be  able  to  control  the  dot  by  applying  pressure 
to  the  non-moving  force  stick. 

You  may  find  it  easier  to  consider  yourself  to  be 
flying  the  grid,  and  to  think  of  the  dot  as  a  target 
toward  which  you  are  flying.   If  the  dot  moves  up,  you 
can  bring  it  back  to  the  center  by  pulling  back  on  the 
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stick.   If  it  moves  down,  push  forward  on  the  stick, 
and  so  on. 

The  testing  session  will  consist  of  seven  runs, 
each  lasting  100  seconds.   Your  score  on  each  run  will 
be  the  total  time  you  are  on  target  divided  by  the  100 
seconds. 

To  compensate  for  a  lack  of  reality  a  high  degree 
of  difficulty  has  been  built . into  the  tracking  task. 
Part  of  this  difficulty  is  the  amount  of  force  required 
in  the  non-moving  force  stick.   It  is  expected  that  your 
arm  will  become  tired:   for  this  reason  you  are  asked  not 
to  touch  the  force  stick  unless  the  blue  light  is  on. 

The  blue  light  will  come  on  five  seconds  before  each 
scoring  run  begins,  and  will  go  off  at  the  end  of  each 
run.   When  the  blue  light  goes  off,  let  go  of  the  stick, 
wait  for  the  pipper  to  stop  moving,  and  then  recenter 
it  if  it  is  not  in  the  center  of  the  grid.   Then  check 
the  size  of  the  scoring  grid  by  moving  the  force  stick 
very  slightly  in  all  directions.   The  green  light  should 
go  out  whenever  the  pipper  reaches   a  distance  of  about 
one  millimeter  from  the  center  of  the  scope. 

When  this  has  all  been  completed,  kick  the  side  of 
the  RED  BARON  two  times .   Shortly  thereafter  the  pipper 
will  jump  to  the  upper  right  corner  of  the  scope  and 
then  come  back  to  the  center.   While  all  this  is  hap- 
pening, you  need  only  sit  back  and  wait  for  the  blue 
light  to  come  on. 
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If  you  have  any  questions  about  procedures  during 
the  test  session,  tap  the  overhead  three  times  to  get 
my  attention.   So  the  code  is:   knock  three  times  on 
the  ceiling  if  you  want  me,  twice  on  the  side  if  you're 
ready  to  go. 

Do  you  have  any  questions? 

The  testing  session  will  begin  shortly." 
Time-Sharing  Phase. 

"During  this  session  you  will  be  asked  to  perform 
an  additional  task  inside  the  RED  BARON.   After  your 
second  run  you  will  see  all  of  the  lights  on  the  left 
side  of  the  display  panel  illuminate.   You  will  be  able 
to  turn  them  off  with  the  toggle  switches  located  next 
to  the  lights.   The  same  switch,  however,  will  not  al- 
ways turn  out  the  same  light.   Before  each  run  begins, 
you  you  must  determine  which  switch  corresponds  to  which 
light.   The  sequence  will  then  remain  constant  until 
the  end  of  that  run.   Before  the  next  run  begins,  you 
will  have  the  same  opportunity  to  learn  the  same  light 
and  switch  sequence. 

So  this  is  what  will  happen  today:   You  will  perform 
only  the  tracking  task  during  your  first  and  second 
runs.   Before  the  third  run  begins,  turn  out  the  lights 
which  have  been  turned  on  remembering  which  switch 
corresponds  to  which  light.   When  the  green  light  comes 
on,  place  all  toggle  switches  in  the  same  position,  and 
don't  touch  them  again  until  the  scoring  run  begins. 
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Recenter  the  scoring  grid,  and  check  it  for  size.   When 
you  are  ready  to  begin,  knock  two  times  as  before.   Then 
just  wait  until  the  blue  light  comes  on. 

When  the  blue  light  comes  on,  you  must  simultaneously 
perform  the  tracking  task  and  turn  out  the  lights .   A 
light  will  come  on  at  the  beginning  of  each  run  and 
either  every  five  seconds  or  every  ten  seconds  thereafter 
until  the  100  seconds  is  completed. 

You  are  asked  to  perform  each  task  as  accurately 
and,  in  the  case  of  the  light-switch  task,  as  rapidly 
as  possible.   You  are  cautioned  that  mistakes  in  the 
light-switch  task  bring  a  heavy  penalty.   You  are  also 
cautioned  that  the  toggle  switches  must  be  moved  gently 
with  only  the  thumb  and  forefinger  or  else  the  scoring 
grid  will  be  dislocated. 

Please  do  not  hesitate  to  stop  the  session  at  any 
time  if  you  have  a  question. 

Do  you  have  any  questions  at  this  time? 

Good  luck. " 
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APPENDIX  B 
LIST  OF  EQUIPMENT 

1.  Analog  Computer  -  Electronics  Associates,  Inc. 

Pace  TR-10  Model  7350 

2.  Tape  Deck  -  Midwestern  Instruments 

ALPHA-W 
3«   Navy  Universal  Counter  -  Dyna  Sciences  Corporation 

CP-929/USN-2^5A 
k.      Timer  -  Hayden  Corporation 

Counter/Timer,  Electromechanical 

5.  Timer  -  LAB-LINE  Instruments,  Inc. 

LAB-CHRON  Timer,  No.  1^02 

6.  Cathode  Ray  Tube  -  Dumont  Corporation 
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